Aim

• Diffusion of a new technology: the web
• Geographers used to be interested in diffusion
• Hägerstrand et al. (1968)
• Passed the torch to economists and sociologists
• Why? Lack of granular data:

Because new digital activities are rarely—if ever—captured in official state data, researchers must rely on information gathered from alternative sources (Zook and McCanless 2022).

Importance

• Guide policies for deployment of new technologies

• Predictions of introduction times for future technologies (Meade and Islam 2021):

  • Network operators

  • Suppliers of network equipment

  • Regulatory authorities

Technological diffusion

Spatial diffusion processes

• As in temporal diffusion models, an S-shaped pattern in the cumulative level of adoption

• A hierarchy effect: from main centres to secondary ones – central places

• A neighborhood effect: diffusion proceeds outwards from innovation centres, first “hitting” nearby rather than far-away locations (Grubler 1990)

Hägerstrand (1965): from innovative centres (core) through a hierarchy of sub-centres, to the periphery

Diffusion of a new digital technology

• Diffusion of an intangible/digital technology [web]

• Map the active engagement with the digital

• Over time, early stages of the internet [1996-2012]

• Granular and multi-scale spatial perspective

Long story short

• Data from the Internet Archive, the oldest web archive

• Observe commercial websites 1996 - 2012 in the UK (.co.uk)

• Geolocation: postcode references in the text

• Timestamp: archival year

• Counts

Web data: The Internet Archive

Web data: The Internet Archive

Long story short

• Data from a Web Archive – The Internet Archive

• Observe commercial websites 1996 - 2012 in the UK (.co.uk)

• Geolocation: postcode references in the text

• Timestamp: archival year

• Counts

Web data: The Internet Archive

• The largest archive of webpages in the world
• 273 billion webpages from over 361 million websites, 15 petabytes of storage (1996 -)
• A web crawler starts with a list of URLs (a seed list) to crawl and downloads a copy of their content
• Using the hyperlinks included in the crawled URLs, new URLs are identified and crawled (snowball sampling)
• Time-stamp

Our web data

• JISC UK Web Domain Dataset: all archived webpages from the .uk domain 1996-2012

• Curated by the British Library

Our web data

• All .uk archived webpages which contain a UK postcode in the web text

• Circa 0.5 billion URLs with valid UK postcodes

20080509162138 | http://www.website1.co.uk/contact_us | IG8 8HD

Data cleaning

• All the archived .uk webpages

• Archived during 1996-2012

• Commercial webpages (.co.uk)

• From webpages to websites:

  - http://www.website1.co.uk/webpage1 and

  - http://www.website1.co.uk/webpage2 are part of the

  - http://www.website1.co.uk

• 1 vs. multuple postcodes in a website

Unique postcodes frequencies, 2000

• Websites with a large number of postcodes: e.g. directories, real estate websites

• Focus on websites with one unique postcode per year

Directory website with a lot of postcodes

Website with a unique postcode in London

Reminder: diffusion mechanisms

• S-shaped pattern in the cumulative level of adoption

• A hierarchy effect: from main centres to secondary ones

• A neighborhood effect: first “hitting” nearby locations

Methods

• Cumulative adoption: Self-starting logistic growth model [nls and SSlogis]

• Descriptive statistics, ESDA & density regressions

• Machine learning framework [random forests]

• Two scales:

  • websites per firm in a Local Authority
  • websites in an Output Area

Diffusion speed

• Spatial heterogeneity

• Not a clear, easy to explain pattern

• Adoption heterogeneity

• Different perceptions of risk and economic returns from new technologies

• Early adopters vs. laggards, leapfrogging

Rank dynamics and diffusion speed

• Spatial heterogeneity

• Expected volatility

Neighbourhood effect: diffusion proceeds outwards from innovation centers, first “hitting” nearby rather than far-away locations (Grubler 1990)

• Spatial dependency (Moran’s I & LISA maps)

• Websites per firm in Local authorities (c. 400)

• Websites in Output Areas (c. 200,000)

Neighbourhood effect

• Spatial dependency
  • Relatively small, consistent over time / scales
  • London hot spot early on
  • At local scale, consistent hotspots over time
  • Granular analysis reveals other hotspots

Hierarchy effect: from main centers to secondary ones – central places

• Gini coefficient

Hieararchy

• Almost perfect polarisation of web adoption in the early stages at a granular level

• Polarisation decreases over time

• More equally diffused at the Local Authority level

Modelling framework

• A hierarchy effect: from main centres to secondary ones

• A neighborhood effect: first “hitting” nearby locations

• S-shaped pattern in the cumulative level of adoption

\[Website\,Density_{t} \sim \color{orange}{Distance\,London} + \color{orange}{Distance\,Nearest\,City} + \color{orange}{Distance\,Nearest\,Retail_{i}} +\\ \color{orange}{Website\,density\,London_{t-1}} + \color{orange}{Website\,density\,Nearest\,City_{t-1}} + \color{orange}{Website\,density\,Nearest\,Retail_{t-1}} +\\ \color{olive}{W*\, Website\,density_{t-1}} +\\ \color{violet}{year_{t}}\]

Modelling framework

• Random forests to predict \(Website\,Density_{i,t}\)

• 2 spatial resolutions:

  • Local Authorities (websites per firm)
  • Output Areas (websites)

• 2 sets of models:

  • All the data ⟹ variable importance
  • Train on all but one region, test on the holdout region ⟹ spatial differences and similarities of diffusion mechanisms

• Space-time sensitive 10-fold CV (CAST)

Models trained on all data

No retail centre variables for the Local Authority models

Variable importance for Local Authorities

Variable importance for Output Areas

Regional similarities for Local Authorities

Regional similarities for Output Areas

Alternative specification: growth, trained on all data

\[Website\,Density\,Growth_{t} \sim \color{orange}{Distance\,London} + \color{orange}{Distance\,Nearest\,City} + \color{orange}{Distance\,Nearest\,Retail_{i}} +\\ \color{orange}{Website\,density\,London_{t-1}} + \color{orange}{Website\,density\,Nearest\,City_{t-1}} + \color{orange}{Website\,density\,Nearest\,Retail_{t-1}} +\\ \color{olive}{W*\, Website\,density_{t-1}} +\\ \color{violet}{year_{t}}\]

No retail centre variables for the Local Authority models

Alternative specification: growth, trained on all data

• Established technological diffusion drivers still apply

  • for a digital technology
  • at local scales

• Geography matters: spatial dependency, urban gravitation

• Some indications of a hierarchical diffusion

• Granular analysis reveals patterns otherwise not visible

• Stability and volatility: leapfrogging, early adopters dropping, but also stable positions

• Spatially consistent mechanisms at local scale

• Heterogeneity increases with resolution

• Things to do: OA?